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CONCEPTUAL MODELS FOR 
INFORMATION AND KNOWLEDGE 


Radu DOBRESCU! 


Abstract. The paper aims to clarify some contradictory aspects that appear in studies on 
Information Science, proposing their approach with the new concept of the General 
Theory of Information (GTI). The main achievement of this theory is to provide a 
comprehensive and relevant definition of information, to whom in the present work is 
added an analysis of the direct relationship is established between information and 
knowledge. Starting from axiological and ontological principles proposed by the initiator 
of GTI, Mark Burgin, various theoretical approaches are analyzed, looking for common 
elements and unifying instances and how to evaluate their properties through adequate 
information measures. As a conclusion, it is argued that many other known directions of 
information theory may be treated inside GTI as its particular cases. 
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1. Introduction 


In order to create a framework to study the conceptual models of information, it is 
mandatory to answer to two main problems related to it. The first one is to define 
what information is and to find what basic properties it has. The second problem 
is how to measure and evaluate information. From the beginning of the 
development of information theory, it was known more how to measure 
information than what information is. Hartley and Shannon gave effective 
formulas for measuring the quantity of information. 


However, without understanding the phenomenon of information, these formulas 
bring misleading results when applied to irrelevant domains. At the same time, a 
variety of information definitions have been introduced. Scientists created a 
diversity of information theories: statistical (Shannon’s theory of communication), 
semantic, algorithmic, qualitative, dynamic and so on. Naturally some of them 
have tried unification of these approaches into a general theory of information 
(GTD. 


One of the first proposals was advanced by Burgin [1]. Burgin’s GTI is built as a 
system of ontological and axiological principles that represent intrinsic properties 
of information and information processes. They reflect the most essential 
properties of information as a natural, social, and technological phenomenon as 
well as regularities of information functioning. 
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2. Formal definitions for Information and Knowledge 


Classical information theory was developed based on the principles set by 
Shannon in his seminal work on the transfer of data from a source to a receiver 
through a communication channel [2]. From the point of view of the general 
theory of communication, information should be seen as an intrinsic feature of a 
system, and not only transmitted information. 


However, without reducing the degree of generality, we will consider the amount 
of information in a receiver system R varies with the changes in that system, i.e. 
information I for a system R is any essence causing changes in the whole system 
R, or in an informational subsystem /F(R) of this system. 


A. Information and its different meanings 


The meaning of “information systems” has been growing in diversity and 
complexity; therefore a first step in the description of a systemic notion of 
information is to identify the meaning of information. 


In the following the word “meaning” is used in its pragmatic sense, i.e. the 
meaning we will looking for the term “information” will be both its conceptual 
definition, as well as its respective practical consequences in the field of 
Information Systems and Informing Sciences. 


The term “information” has been widely and increasingly used, but not always 
with a clear idea about its meaning. The word “information” is one of the most 
used, and much abused, words. Different scientific disciplines and engineering 
fields provide diverse meanings to the word, which is becoming the umbrella of 
divergent, and sometimes dissimilar and incoherent homonyms. When concepts 
are not clear, the use of homonyms might be intellectually and pragmatically 
dangerous. 


We will try, here, to make an initial step, attempting first a conceptual definition 
of information that underlines its two aspects: subjective and objective. 


Information has been frequently defined as “interpreted data” and, as such, the 
same data might cause different interpretations. An interpretation is, by its own 
nature, subjective. Consequently, it is easy to conclude that according to this kind 
of definition there is no IS without a subjective sub-system, i.e. any IS should 
have at least two subsystems: an objective (mechanical and/or electronic data 
processing subsystem) and a subjective one (biological/human data/information 
processing: a user, a manager, etc). 


One can consider that the objective information is external to human beings, but is 
created by them. What might be called objective information is a representation of 
the real information, which always is a subjective one in its origin and essence. 
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Therefore, the conclusion is evident: information is generated inside the mind of a 
person, a subject. It is not an objective entity independent of any person. It is 
dependent on the person where it is generated by the data stimulus, as well as on 
his/her individual experience. 


Callaos [3] defined information as “decision-relevant data”, which makes of it 
something requiring a special kind of subjectivity, a strict subjectivity that exclude 
the possibility of trans-subjectivity, due to the personal nature of “decision” and 
“relevant decision”, because decisions are always subjective, and relevancy is 
always related to a given subject. 


Consequently, we can observe that subjective reception of the data is a necessary 
condition for in-formation generation, but it is not a sufficient one. It is important 
to find out the additional conditions that data should comply with, in order to be 
informative. So, we can draw the following conclusions: 


e a datum is a “given” thing, not any “given” thing, but the one that makes a 
difference 


e information is a cognitive content, not any cognitive content, but the one 
related to the association of data and a relevant question, be it implicit or 
explicit 


e data and information are like the two sides of the same coin: datum is the 
objective side of the coin and information is its subjective side. This 
relation might be seen as analogous to the relation between the signifier 
(the objective/material side of a sign) and the signified (its 
subjective/mental side), in semiological terms. 


B. Knowledge and its different meanings 


In the most used interpretation, knowledge is what makes people able to perform 
actions, and in this way we can say that knowledge is what makes organisations 
(structures, systems) work. Viewing knowledge as a critical asset implies efforts 
to utilise, improve, and disseminate knowledge in the organisation. 


Going back to the origin the word “knowledge”, one early epistemological 
philosopher who has put forward a distinct approach to knowledge is Aristotle. 
Aristotle builds on Plato’s definition of knowledge, which is said to be true, 
justified beliefs. In a strict sense this form of knowledge is theoretical-scientific 
knowledge (episteme) and represents knowledge about something. Techne is 
another form of knowledge that Aristotle presents, which concerns skills and 
ability to do something. While episteme comes from true, justified beliefs, techne 
emanates from what we do, i.e. individuals’ actions. Thus techne is connected to a 
pragmatic dimension of knowledge. 
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The third form of knowledge Aristotle talks about is phronesis, which is also a 
form of practical knowledge that has its starting-point in knowledge-in-action, just 
as techne is. Phronesis concerns practical sense making and aims at enhancing 
humans’ well being. Hereby, phronesis includes both political and ethical 
dimensions. 


If we look at these two forms of knowledge, they have been used as foundations 
for several different epistemological approaches. For example, Popper asserts that 
knowledge in its objective (and for science the only interesting) sense is 
knowledge without anyone who knows, i.e. knowledge without a knowing subject 
[4]. This view of knowledge might be equivalent to episteme. Techne, on the other 
hand, represents the pragmatic tradition of knowledge and one author who has 
asserted this approach is Peirce [5]. 


The question is whether knowledge is always subject-related, or if knowledge 
could take the shape of an object. The interpretation, understanding, and use of 
knowledge are different knowledge-focused processes. In this way knowledge is 
often, if not always, related to a specific context. 


Nevertheless, knowledge is not exclusively situated or completely bound to its 
context. What needs to be emphasised is that knowledge descriptions should not 
be considered as knowledge per se, it is just representations of someone’s 
personal knowledge. The translation of objective knowledge to subjective 
knowledge within the receiver always requires interpretation, understanding, and 
the use of language. 


C. The relations between information and knowledge 


Information and knowledge are important part of the world surrounding us, as 
well as part of ourselves. There are a large number of definitions of information in 
relation with knowledge. Drucker says that information is data endowed with 
relevance and purpose. The conversion of data into information thus requires 
knowledge, and knowledge is specialized [6]. 


Turban et al. have a similar definition, and mean that one of the primary goals of 
an information system is to process data into information and knowledge [7]. 


These authors refer to data as elementary descriptions of things, events, activities, 
and transactions that are recorded, classified, and stored but not organized to 
convey any specific meaning. They treat information as data that have been 
organized so that they deliver meaning and value to the recipient. 


Finally, they say that “knowledge is said to consist of data or information that 
have been organized and processed to convey understanding, experience, 
accumulated learning, and expertise as they apply to current problem or activity”. 
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The relations between data, information, and knowledge are often used in order to 
describe the meaning of and relation between the three words. 


In spite of the great number of definitions that are available in order to explain the 
meaning of information (and data, and knowledge), the notion is still not very 
clear. 


One can actually say that the definitions stated above are more mystifying than 
clarifying. The major basis of this criticism is that the definitions leave us 
uncertain as to when information, data and knowledge exist (their ontological 
determination). 


But not much has been done regarding the notion of meaning by itself, whatever 
the information associated to it or the system managing it. It is this aspect of 
relations linking information and meaning we are interested in. 


We propose that there can be some common ground for most types of meanings, 
and that this common ground can be explicated by defining a basic system 
managing the information and the associated meaning. 


Therefore, let's formulate some characteristics relative to the notion of meaning 
we are trying to conceptualize, in order to bring up a "systemic aspect" for the 
framework of the relations between an information processing system and 
information received by this system. 


1) A meaning is associated with information which is incident on a system 
capable of processing the information. 


2) Meaning is generated because the information processing system possesses a 
constraint linked to its nature. 


3) A meaning is generated because the incident information has a connection with 
the constraint of the system. 


4) A meaning is meaningful information relatively to the constraint of the system. 


5) The meaningful information is going to participate to the determination of an 
action that the system is to implement in order to satisfy its constraint. 


Based on these five characteristics, let now define a mixed concept of the 
informational system IF(R) that dissolves the objective-subjective dichotomy and 
focus on what relates and communicates them, i.e. what is common to both of them. 


Definition 1. A subsystem IF(R) of the system R is called informational system of 
R if F(R) contains informational elements place it not just in the subject, or in the 
object, but in both of them and in what relates them. 


In the next section are presented the basic principles that allow considering /F(R) 
compatible with the requirements of Burgin’s general theory of information. 
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These principles explain how to evaluate information and what measures of 
information are necessary in this aim. 


3. Basic Principles of the General Theory of Information 


According to the ontological principles ([8]), it is natural to assume that measure 
of information is determined by the results that are caused by reception of the 
corresponding portion of information. 


Principle 1 (the ontological principle). An arbitrary complex system R has, as a 
rule, different informational subsystems. Fixing one of these subsystems, we 
determine what information for R is and changing our choice of IF(R) we change 
the scope of information entities. 


This principle implies that for a complex system there are different kinds of 
information. Each informational system determines a specific kind of information. 
For example, information that causes changes in the system of knowledge is 
called cognitive information. 


Distinction between knowledge and cognitive information implies that transaction 
of information does not give knowledge itself. It only causes such changes that 
may result in the growth of knowledge. 


Principle 2 (the relational principle). Considering (R, L, E) a triadic complex 
where R is a system which receives information, EF is the environment of this 
system and L represent different links between R and E, there are three structural 
types of measures of information: external, intermediate, and internal. 


An internal information measure reflects the extent of inner changes caused by I. 
An intermediate information measure reflects the extent of changes caused by J in 
the links between R and E. An external information measure reflects the extent of 
outer changes caused by J, 1.e., the extent of changes in E. 


Principle 3 (the axiological principle). There are three constructive types of 
measures of information: abstract, realistic, and experiential. An abstract 
information measure is determined theoretically under general assumptions. 


A realistic information measure is determined as subject to realistic conditions. 
Quality of information is an example of such measure. An experiential 
information measure is obtained through experimentation. 


According to this principle a unique measure of information exists only for 
oversimplified system. Any complex system R with a developed informational 
subsystem /F(R) has many parameters that may be changed. So, such systems 
demand many different measures of information in order to reflect the full variety 
of these systems properties as well as of conditions in which these systems 
function. 
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Principle 4 (the communication principle. Measure of information I, which is 
transmitted from a source of information $ (Sender) to a system R (Receiver), 
depends on interaction between S and R. A system R receives information I only 
if some carrier C of J transmits this information through some channel CH. 


This principle corresponds to the traditional communication model defined by 
Shannon. Communication process as a total event has been a subject of many 
studies, and consequently many models have been suggested, but in all cases a 
measure of information transmission reflects a relation between measures of 
information that is accepted by the system R in the process of transmission and 
information that is presented by S in the same process. 


4. Theoretical approaches in the Information Science 


We consider Information Science as a most appropriate term than Information 
Theory to cover the multifaceted models for information and knowledge. 


In the same time, we consider that the various developed theoretical approaches in 
Information Science are particular cases of the General Theory of Information 
because they explicitly or implicitly consider treat information from the functional 
point of view as a kind of transformations in a system. 


To prove this, we find in each case a relevant informational system /F(R) and 
demonstrate that in each of these approaches information is what changes this 
system. 


The Statistical Information Theory 


The statistical approach is now the most popular direction in the information 
sciences. It is traditionally called Shannon's information theory or, as it was at 
first named by Shannon, the theory of communication. 


It is a mathematical theory formulated principally by the American scientist 
Claude E. Shannon to explain aspects and problems of information and 
communication. 


The basic problem that needed to be solved, according to Shannon, was the 
reproduction at one point of a message produced at another point. He deliberately 
excluded from his investigation the question of the meaning of a message, 1.e., the 
reference of the message to the things of the real world. 


While the statistical theory of information is not specific in many respects, it 
proves the existence of optimum coding schemes without showing how to find 
them. 


For example, it succeeds remarkably in outlining the engineering requirements of 
communication systems and the limitations of such systems. 
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In the statistical theory of information, the term information is used in a special 
sense: it is a measure of the freedom of choice with which a message is selected 
from the set of all possible messages. 


Information is thus distinct from meaning, since it is entirely possible for a string 
of nonsense words and a meaningful sentence to be equivalent with respect to 
information content. 


Shannon proposes also a measure unit: information is measured in bits. One bit is 
equivalent to the choice between two equally likely choices. A special measure 
that is called quantity of information of a message about some situation is defined 


N 
by the informational entropy H(p), H(p)=—>) p, log p, where N is the number of 


i=l 


possible cases of the situation in question and p;is the probability of the case i. 


So, information is considered as elimination of uncertainty, i.e., as a definite 
change in the knowledge system that is the informational system /F(R) of the 
receptor of information. Since the information content is, in general, associated 
with a source that generates messages, information is often called the entropy of 
the source. 


To this day, there is no measure in information theory that is as well-supported 
and as generally accepted as Shannon's quantity of information. 


On the other hand, Shannon's work is rightly seen as lacking indications for a 
conceptual clarification of information. As a result, other directions have been 
suggested in information science. 


The Semantic Theory of Information 


The semantic theory of information tries to encompass semantic aspects of 
information because some experts in information sciences consider meaning as 
the essence of information. This approach is based on the assumption that every 
piece of information has the characteristic that it makes a positive assertion and at 
the same time makes a denial of the opposite of that assertion. 


In the semantic information theory, transformations of such informational system 
as thesaurus T, or system of knowledge, are treated as information. The founders 
of this approach, Bar-Hillel and Carnap build it as a logical system [9]. Thus, the 
informational systems in this theory are sets of logical propositions. These 
propositions are used to describe and represent the state of an arbitrary system. 


The corresponding measure of information is defined for a separate proposition as 
the probability that this proposition is a part of the description of the real state of a 
system under consideration. Thus, information causes change in knowledge about 
this state. 
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The Qualitative Information Theory 


In the qualitative information theory, the following definition is given: 
Information is a transformation of one communication of an information 
association into another communication of the same association [10]. So, here the 
informational system /F(R) is some information association and information is a 
transformation of R. 


The Algorithmic Theory of Information 


The principal aim of the algorithmic theory of information is the development of 
more realistic concepts of randomness and probability than those that are in the 
traditional theory of probability. This new approach was suggested in the works of 
three authors: Solomonoff [11], Kolmogorov [12], and Chaitin [13]. The main 
idea was that algorithms play a leading role in many processes, including people 
behavior. Consequently, theory of algorithms has to be central in a study of such 
processes. 


In the algorithmic information theory, there are two kinds of measures of 
information. The first one is called the entropy, or information content, or 
complexity of finite constructive objects (like strings of symbols). Such 
algorithmic measure of information is defined to be the number of bits (or 
symbols) needed to specify the object in question so effectively that it can be 
constructed. 


This measure is called the Kolmogorov complexity of x and is denoted by K(x). 
By the definition, this is the measure of such a transformation as a construction of 
some object. It is an external measure of information that acts on such 
informational system /F(R) as thesaurus, or the system of knowledge. 


Knowledge in this system is represented by strings of symbols (texts) that have 
meaning. It is possible to define a more general measure of information of the first 
type. It is called a generalized Kolmogorov complexity measure. 


Generalized Kolmogorov complexity measures reflect a measure of resources 
needed to construct the object in question. Generalized Kolmogorov complexity 
measures are dual to computational complexity measures. 


The Pragmatic Theory of Information 


The concept of pragmatic information was introduced by Ernst Ulrich von 
Weizsiacker [14] and further developed by von Lucadou [15], Kornwachs [16] and 
some others. This concept relies on the two notions: firstness (“Erstmaligkeit”) or 
originality or novelty, and confirmation (“Bestatigung”) or redundancy to already 
known. Weizsiécker suggests that a message that does nothing but confirm the 
prior knowledge of a receiver will not change its structure or behaviour. 
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Thus, with confirmation up to 100%, a message gives no pragmatic information. 
On the other hand, a message providing only original/novel material completely 
unrelated to any prior knowledge also will not change structure or behaviour of 
the receiver, because the receiver will not understand this message. 


Thus, with firstness up to 100%, a message gives no pragmatic information. Only 
a relevant mixture of firstness and confirmation allows the receiver to get 
pragmatic information from the message. 


Thus, pragmatic information, according to Weizsacker, is related to changes of 
structure or behaviour of the receiver. In such a way, pragmatic information acts 
on the observed system. This entirely corresponds to the concept of information in 
the general theory of information. 


However, the general theory of information allows us to go further and to consider 
two opportunities of change. In one case, behavior of the system under 
observation changes. It corresponds to an action on the behavioral informational 
system. In the other case, changes only information that we get from the system 
under observation. It corresponds to an action on the _ representational 
informational system. 


The Utility Theory of Information 


In the utility theory of information [17], the measure of information is called the 
quality of information. It is defined for mission oriented systems R. If I is some 
portion of information, then the quality of this information is equal to the caused 
by I change of the probability p(R, g) of achievement of a particular goal g by the 
system R. 


If we consider objective probability, then the corresponding informational system 
is the state space of the world in which system R functions. If we consider 
subjective probability, then the corresponding informational system is the belief 
space in which probabilities for different events involving system R are 
represented. In both cases, information appears as a change in the corresponding 
informational system. 


The Dynamic Theory of Information 


The principal aim for the development of the dynamic theory of information was 
investigation of biological information and information processes in living 
systems [18]. 


In the dynamic information theory, the notion of information is taken as a basic 
one. Information is considered as a sequence of two operations: choice of one 
alternative from a collection of possible alternatives and saving the chosen 
alternative. Thus, information is also treated as an action causing transformations. 
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The Evolutionary Information Theory 


Evolutionary information theory [19] is a constructive approach that studies 
information in the context of evolutionary processes, which are ubiquitous in 
nature and society. It corresponds to Burgin interpretation of the Dynamic Theory 
of Information as component of the General Theory of Information, which built 
several measures of evolutionary information based on mathematical models of 
evolutionary computations, machines and automata. 


The first class of measures is the evolutionary information size of symbolic 
objects relative to classes of automata or machines. 


The second class of measures for evolutionary information in symbolic objects is 
studied by introduction of the quantity of evolutionary information about 
symbolic objects relative to a class of automata or machines. 


As main applications, this theory allows to study relations between evolutionary 
information and quantum information and to study biological information in 
general and genetic information in particular from the evolutionary information 
perspective. 


The Integrated Information Theory. 


Integrated information theory (IIT), despite its name, has nothing to do with the 
term “information” as it has been previously discussed, but indirectly refers to 
knowledge, specifically how someone becomes aware (conscious) of what he 
know. IIT attempts to explain what consciousness is and why it might be 
associated with certain physical systems. 


Given any such system, the theory predicts whether that system is conscious, to 
what degree it is conscious, and what particular experience it is having. 


According to IIT, a system's consciousness is determined by its causal properties 
and is therefore an intrinsic, fundamental property of certain causal systems. IIT 
was proposed by neuroscientist Giulio Tononi in 2004, and has been continuously 
developed over the past decade. The latest version of the theory, labeled IIT 3.0, 
was published in 2014 [20]. 


IIT argues that the existence of an underlying causal system with these same 
properties offers the most parsimonious explanation. The properties required of a 
conscious physical substrate are called the "postulates". 


A "physical system" is taken to be a set of elements, each with two or more 
internal states, inputs that influence that state and outputs that are influenced by 
that state (neurons or logic gates are the natural examples). 


Given this definition of "physical system", the postulates are: 
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- Intrinsic existence: To account for the intrinsic existence of experience, a 
system constituted of elements in a state must exist intrinsically (be actual). 


- Composition: The system must be structured: subsets of the elements 
constituting the system, composed in various combinations, also have cause-effect 
power within the system. 


- Information: The system must specify a cause-effect structure and a 
specific set of specific cause-effect repertoires. A cause-effect repertoire 
characterizes in full the cause-effect power of a mechanism within a system by 
making explicit all its cause-effect properties. 


- Integration: The cause-effect structure specified by the system must be 
unified: it must be intrinsically irreducible to that specified by non-interdependent 
sub-systems obtained by unidirectional partitions. Intrinsic irreducibility can be 
measured as integrated information (“big phi” or @, a non-negative number), 
which quantifies to what extent the cause-effect structure specified by a system’s 
elements changes if the system is partitioned (cut or reduced) along its minimum 
partition (the one that makes the least difference). This postulate also applies to 
individual mechanisms: a subset of elements can contribute a specific aspect of 
experience only if their combined cause-effect repertoire is irreducible by a 


minimum partition of the mechanism (“small phi” or? ). 


- Exclusion: The cause-effect structure specified by the system must be 
definite: it is specified over a single set of elements - the one over which it is 


maximally irreducible from its intrinsic perspective (®"” ). This implies that a 
mechanism cannot specify a cause-effect repertoire at a particular temporal grain, 
and additional effects at a finer or coarser grain, otherwise the differences a 
mechanism makes would be counted multiple times. 


Let’s finally note that in order to apply the IIT formalism to a system, its full 
transition probability matrix (TPM) must be known. The TPM specifies the 
probability with which any state of a system transitions to any other system state. 


5. Conclusions 


It is necessary to remark that the modern approaches consider information as 
some kind of knowledge. More exactly, in modern information theory, a 
distinction is made between structural-attributive and functional-cybernetic types 
of theories. The representatives of the former approach conceive information as 
structure, like knowledge or data, variety, order, and so on; members of the latter 
understand information as functionality, functional meaning or as a property of 
organized systems. The general theory of information treats information from the 
functional and, more exactly, dynamic perspective. A_ structural-attributive 
interpretation does not represent information itself but relates to information 
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carriers. Consequently, structural-attributive types of information theories are also 
included in the scope of the general theory of information because structures and 
attributes are represented in this theory by informational elements and _ their 
properties and systems. 


As conclusion, we can observe that the general theory of information makes it 
possible to solve many open problems related to information and achieve a new 
profound understanding of information as a natural, technological, and social 
phenomenon. It provides for systematization of almost all other approaches in 
information sciences eliminating many of their shortcomings. 


In particular, a new, more adequate definition of information is obtained. In 
addition, it allows us to discover new types of information: cognitive, 
conditioning, and regulative information. Cognitive information gives knowledge 
and thus, it is what people know under the name of information. 


Two other types are new and help to understand many phenomena in system 
functioning. This discovery of new types of information makes possible to 
determine the role of information for system functioning. 


Conditioning information is basic for any system from the perspective of its inner 
evolution. Regulative information is basic for any system from the perspective of 
its interaction with environment. Cognitive information appears only on higher 
levels of system development. 


The role of cognitive information increases with the development of a system and 
becomes decisive on some level of this development. 
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